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ABSTRACT 

Objective: To estimate the effect of online adverts on 
the probability of finding online cognitive behavioural 
therapy (CBT) for depression. 
Design: Exploratory online cross-sectional study of 
search experience of people in the UK with depression 
in 2011. (1) The authors identified the search terms 
over 6 months entered by users who subsequently 
clicked on the advert for online help for depression. (2) 
A panel of volunteers across the UK recorded websites 
presented by normal Google search for the term 
'depression', (iii) The authors examined these websites 
to estimate probabilities of knowledgeable and naive 
internet users finding online CBT and the improved 
probability by addition of a Google advert. 
Participants: (1) 3868 internet users entering search 
terms related to depression into Google. (2) Panel, 
recruited online, of 12 UK participants with an interest 
in depression. 

Main outcome measures: Probability of finding 
online CBT for depression with/without an advert. 
Results: The 3868 users entered 1748 different search 
terms but the single keyword 'depression' resulted in 
two-thirds of the presentations of, and over half the 
'clicks' on, the advert. In total, 14 different websites 
were presented to our panel in the first page of Google 
results for 'depression'. Four of the 14 websites had 
links enabling access to online CBT in three clicks for 
knowledgeable users. Extending this approach to the 
10 most frequent search terms, the authors estimated 
probabilities of finding online CBT as 0.29 for 
knowledgeable users and 0.006 for naive users, 
making it unlikely CBT would be found. Adding adverts 
that linked directly to online CBT increased the 
probabilities to 0.31 (knowledgeable) and 0.02 (naive). 
Conclusions: In this case, online CBT was not easy to 
find and online adverts substantially increased the 
chance for naive users. Others could use this approach 
to explore additional impact before committing to 
long-term Google AdWords advertising budgets. 
Trial registration: This exploratory case study was a 
substudy within a cluster randomised trial, registered on 
http://www.clinicaltrials.gov (reference: NCT01 469689). 
(The trial will be reported subsequently). 



ARTICLE SUMMARY 



Article focus 

■ In 2011 in the UK, what is the chance of finding 
online CBT for depression? 

■ Does online advertising increase that chance? 



Key messages 

■ Online CBT for depression is recommended by 
the National Institute for Health and Clinical 
Excellence for people with depression, but we did 
not know how easy it is to find or if online 
adverts help find it. 

■ People with depression are unlikely to find online 
CBT by chance: the probabilities of finding online 
CBT using Google was 0.29 for knowledgeable 
users and 0.006 for naive users. Adding adverts 
that linked directly to online CBT increased the 
probabilities to 0.31 (knowledgeable) and 0.02 
(naive). 

■ This method of assessing probability of finding 
topics should be used before committing to long- 
term online advertising. 

Strengths and limitations of this study 

■ The results are likely to be typical for anyone with 
depression in the UK in 2011. The approach of 
identifying typical search terms and simulating 
knowledgeable and naive user experience, with 
and without adverts, is generalisable to other 
conditions and populations. 

■ Search results will change over time. 



BACKGROUND 

Less than 60% of people with diagnosable 
depression or anxiety seek formal help from 
practitioners; this represents a significant 
treatment gap. 1 The remainder may access 
informal care and support, alternative thera- 
pies, private arrangements such as counsel- 
ling, use of the voluntary sector and the 
internet and other sources of information. 

Nearly one in five British internet users 
search for information related to mental 
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health. 2 Among younger people, this may be much 
higher. Reasons cited included anonymity, the amount 
of information available and easy access. 3 5 The level of 
searching for information on depression online has 
increased. In 2000, depression was not in the 10 most 
frequently searched for topics, but by 2005, it was the 
fifth most common topic. 5 

Others have investigated quality of health information 
websites, including those for depression, 6 and attempted 
to rate the quality of these websites. 7-9 Patients 
searching for health information online may be unable 
to find what they are looking for, 10 possibly due to 
suboptimal search strategies. 11 Most people follow-up 
between the first three and 10 results. 3 12 We have not 
found studies of how users seek online help for depres- 
sion but most people searching for health information 
start with a search engine. 10 Google's share of the search 
market is debatable 13 14 but we can assume that 
a majority in the UK searching for health information 
will use Google. Search results may now be personalised 
according to user's location 15 and previous searches. 16 17 
Some regard this personalisation as intrusive and this 
has led to the development of software to hide user 
characteristics. 18 Search results will also change over 
time because of adjustment of Google's databases to the 
changing pattern of websites. This means that there is no 
standard response to a Google search; however, we can 
'average' the experience of several users to explore ease 
of finding given topics at any time. 

Although not everyone searching online using terms 
such as depression is depressed, we need to make sure 
that the best support is available for those who are. 
Online cognitive behavioural therapy (CBT) is self-help 
treatment recommended by the National Institute for 
Health and Clinical Excellence (NICE) in the UK for 
mild to moderate depression. 19 The range of resources 
includes licenced (paid for) sites such as Beating the 
Blues 20 and free access websites providing access to CBT 
life skills resources (eg, Living Life to the Full (LLTTF) 21 
and MoodGYM). 22 

Given the NICE recommendation and the predomi- 
nance of Google, it would be reasonable to expect 
websites delivering access to online CBT to have relatively 
high profiles and be easily visible to users on a Google 
search. We were struck by the large geographical variation 
within the UK in the uptake of LLTTF. The highest rate 
(in Kirkwall, Scotland) was 15 times that of the lowest 
(Wigan, England) . Such variation could not be explained 
by variation in prevalence of depression and was most 
likely explained by lack of awareness of online therapies 
for depression. We have been exploring ways of increasing 
awareness. 23 One method of doing this was to use online 
advertising, in particular, Google AdWords. 24 25 

Those paying for an AdWords campaign set up one or 
more adverts (see figure 1 for example) and enter 
keywords to help determine when the advert is shown. 
AdWords determine if an advert is relevant to a given 
user's search based on (in AdWords terminology) exact 



Free help for depression 
NHS recommended sites 
Free and easy to use 

onlinehelpfordepression.org.uk 



Figure 1 Google advert. 

matches, broad matches and phrase matches with the 
user's entered search term. 26 AdWords displays adverts 
as a sponsored link, either at the top of the list of search 
results or in the right-hand panel, depending on the 
phrase entered, the price you offer per advert and bids 
from competing adverts, and (if requested) by estimated 
location of the user. Google AdWords may be embedded 
in searches in other websites (the Google Display 
Network). 27 

However, diverting users who would have found CCBT 
sites without the aid of adverts, to finding it via an advert, 
is a waste of resources. In this case study, we reviewed the 
likely 'search experience' using Google of people 
searching using the term 'depression' to estimate the 
chance of finding online CBT and the improved proba- 
bility by adding an online advert. 

METHODS 

Ethics and trial registration 

This was an exploratory substudy within a pilot cluster 
randomised trial. The whole study was approved by the 
Plymouth NHS Research Ethics Committee (reference. 
11/H0203/8), is registered on http://www.Clin- 
icalTrials.gov (reference: NCT01469689) and will be 
reported subsequently. 

Setting and population 

We reviewed search terms entered by 3868 internet users 
in the UK who used the Google search engine for terms 
related to depression and who clicked our presented 
Google advert (figure 1) between 17 April 2011 and 31 
August 2011. 

We enabled 126 keywords in our AdWords campaign 
and allowed exact matches, broad matches and phrase 
matches. Those clicking on the advert were taken to 
a website which asked them to confirm their location 
(postcode area) and to complete a widely used depression 
rating questionnaire (PHQ9 28 ) and then offered links to 
four sites addressing low mood in different ways. Our 
main objective was to offer a link to online CBT, in 
particular, LLTTF and MoodGYM. 29 The NHS research 
ethics committee that considered our proposal asked us 
to add a link to Samaritans. 30 (Samaritans is a secular UK 
charity aimed at providing emotional support to anyone 
in emotional distress or at risk of suicide mainly through 
their 24/7 confidential telephone helpline.) To keep 
symmetry in the exit page of the website, we added NHS 
Choices (non-CBT information about depression) . 31 The 
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Figure 2 Schematic of methods. 
CBT, cognitive behavioural 
therapy. 
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order of links presented on the page was randomised 
within each row, the top row always being the two CBT 
sites and the bottom row Samaritans and NHS Choices. 

Search terms used 

We reviewed the presentation of our advert and the search 
terms entered by users (who subsequently clicked on our 
advert) to understand better what terms people may use 
and how many may search directly for online CBT. 

Exploring the results of normal Google search 

Given that the results of Google searches are to some 
degree personalised and therefore will vary by computer 
used, to assess which websites would be found by users, we 
recruited a user panel of 12 people. We contacted 
a sample of previous users of TTTTF and through an 
AdWord's advert (using the keyword depression) sought 
volunteers in our target areas (12 of 121 postcode areas in 
Britain) . 23 These people, in a range of locations, are likely 
to have had search history including previous searches for 
depression and so would be presented with different 
Google search results. These results would be represen- 
tative of those obtained by people in our target areas 
searching for depression-related terms. Although we 
tried to target our study postcode areas in recruiting the 
panel, there was some 'leakage'. Nine of 12 were in target 
areas (Darlington, Tancaster, Orkneys, London SW, 
Kingston and Liverpool) and three were in other areas 
(Twickenham, Gloucester and Dorchester) . We asked our 
user panel to search Google using the single word 
'depression' and email us a screen shot of the first page of 
results (usually 10 results). We also wanted to explore the 
variability of search results by different entered terms. We 
felt unable to ask our panel to carry out and report more 
than one search so two of the authors searched for the 
next nine most entered search terms (table 2). 



Estimating the chance of finding online CBT from websites 

We assessed the websites found by people using the term 
'depression' to see how easy it was to find online CBT. 
We first assumed a knowledgeable user and assessed if 
online CBT (in this case specifically LLTTF or 
MoodGYM) could be reached in three clicks by someone 
who knew that online CBT sites existed. For those 
websites where online CBT could be reached in three 
clicks, we calculated the probability that someone (ie, an 
'extreme' novice) who made completely random choices 
would find online CBT purely by chance. To do this, we 
noted how many choices were involved at each level of 
the shortest pathway to online CBT to calculate an 
approximate probability of reaching online CBT. 

Model to estimate probabilities of finding online CBT 

We used a number of assumptions (see results) to esti- 
mate the probability of finding online CBT with and 
without Google adverts. Figure 2 shows a schematic of 
the samples used and how information from different 
stages was used to calculate the probabilities. 

RESULTS 
Search terms 

By 31 August 2011, Google had presented our advert 
299 369 times to various Google searches that included 
combinations of our keywords, resulting in 3868 'clicks'. 
Google reported the 1748 most frequently entered search 
terms entered by users which were responsible for 71 124 
(24%) of the presentations. Examples are shown in table 
1. Only four (0.2%) search terms of 1748 included either 
the words cognitive, behavioural or behavioural and this 
resulted in 72 presentations (one in 1000) of our advert. 

Ordering the search terms by the number of resultant 
clicks on our advert, we selected the 10 top terms (table 
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Table 1 Examples of search terms as entered (including spelling mistakes and typing errors) by users for which our Google 
Advert was displayed and for which the user clicked on the advert 



am i depressed 
anxiety help in blackpool lanes 
best cures for depression 
bipolar help free 

cancer sufferer with depression how to help 
cipralex blog 

coping with anxiety and depression 

cures for stress 

deperssion simtons 

depression about dying 

depression and sleep 

depression clinics UK 

depression help free 

depression in men 

depression pacing up and down 

depression sign 

depression test and help 

depression working 

does acupuncture help depression 

episodes of bipolar disorder 

food for depression 

free on line suport for depreshon 

getting help for depression 

help for depressed people 

help friend with depression 

help with depession 

herbal remedies for depression 

how can i help someone who has depression 



how does one get depressed 
how to deal with depression how to get over 
depression and loneliness 
how to know if your depressed 
how to treat depression and trauma 
i need help depression 
information on depression and anxiety 
living with depression 
manic depressive 
mental health helplines 
mood stabilisers 
neediness and depression 
online depression test 
paradox tablets for good mental health 
post traumatic stress disorder 
reasons for depression 
Samaritans free 
self help depression 
sign of depression 
stop depression 
suicidal thoughts 
suicide websites 
take a depression test 

the ultimate guide to getting over depression 
treatments for depression nhs 
what activities can help with depression 
what is depression disorder 
where do i get help for depression woman & depression 



Systematic sample of every 30 of 1748 entries ordered alphabetically. 



2). Although these 10 terms only resulted in 1407 
(36%) of the clicks, other entered terms, including the 
terms in table 2 with additional words or misspelled, 
were likely to have resulted in similar Google search 
results. 

AdWords matched the entries made by users with the 
126 keywords using broad matches, exact matches and 
phrase matches. We reviewed the most popular keywords 
as listed by AdWords and the full search terms that were 
entered by users. 'Depression' as a single keyword in our 



Table 2 The top 10 entered search terms showing the 
number of resultant clicks on our advert 




Resultant clicks 


Search term entered 


on our advert 


depression 


886 


depression help 


127 


help with depression 


71 


depression symptoms 


60 


self help for depression 


58 


Samaritans 


57 


depression test 


51 


dealing with depression 


36 


coping with depression 


33 


depression self help 


28 





Google AdWords campaign resulted in two-thirds of the 
advert presentations and over half of the clicks. 

Websites offered by Google search 

Given that Google search results are to some degree 
personalised by location and previous searches, we 
wanted to know what a 'typical' search result would be. 
As the single word 'depression' was the most frequent 
search, we asked our UK user panel to search with that 
single word. The variability of search results using the 
single term depression was relatively small; only 14 sites 
in total were presented to the 12 users (table 3). 
(If searches were exactly the same, there would only be 
10 sites, but if searches were all completely different, 
there would be 120.) Six sites were seen by all 12 users. 
The NHS Choices appeared in the top three for all. 

Finding online CBT from Google displayed websites 

One researcher (LG) explored each of the 14 websites in 
table 3 to see if online CBT could be found within three 
clicks. Four of the website had links enabling access to 
either MoodGYM or LLTTF in three clicks (table 3). 

We then estimated the probability of finding online 
CBT on these four websites by chance, that is, for an 
'extreme naive' user. Table 4 is constructed knowing 
pathways that will lead to online CBT and while following 
that path, counting the total number of choices 
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Table 3 Websites (with hyperlinks) identified by 12 
participants searching Google using the term 'depression': 
showing the number of users for whom the website was on 
first page of results 





No. of 


No. of 




users where 


users where 




website on 


website was 




first page 


in first three 


Website 


of results 


results 



Sites from which a knowledgeable user could find 

MoodGYM or LLTTF within three clicks 
NHS Choices 12 12 

http://www.nhs.uk/Conditions/Depression/Pages/ 
Introduction. aspx 

Net Dr 12 3 

http://www.netdoctor.co.uk/depression/index.shtml 
Patient.co.uk 1 1 0 

http://www. patient. co. uk/health/Depression . htm 
RC Psych 12 0 

http://www.rcpsych.ac.uk/mentalhealthinfoforall/ 
problems/depression/depression. aspx 

Sites from which a knowledgeable user could not find 

MoodGYM or LLTTF within three clicks 
BBC Health 12 0 

http://www.bbc.co.uk/health/emotional_health/ 
mental_health/disorders_depression.shtml 
Depression Alliance 12 2 

http://www.depressionalliance.org/ 
Mind 12 3 

http://www.mind.org.uk/help/diagnoses_and_conditions/ 
depression 

Wikipedia 10 10 

http://en.wikipedia.org/wiki/Depression 

1 on 1 Health 2 0 

http://www.1on1health.com/web/info/depression/english/ 

home/Topic-Home 

Fighting Depression 2 0 

http://www.fightingdepression.co.uk/ 

Depression UK 2 0 

http://www.depressionuk.org/ 

Clinical Depression 1 0 

http://www.clinical-depression.co.uk/ 

Depressed Test .com 7 0 

http://www.depressedtest.com/ 

Mental Health Foundation 1 0 

http://www.mentalhealth.org.uk/help-information/mental- 

health-a-z/D/depression/ 

Websites are presented grouped into four websites where 

a knowledgeable user (ie, someone who knew that online CBT 

existed and was searching for it) could find CCBT within three 

clicks and 10 websites where they could not. 

CBT, cognitive behavioural therapy; LLTTF, Living Life to the Full. 



available. For example, on the Royal College of Psychi- 
atrists site, the 'landing' depression web page from 
Google currently has seven main choices within the body 
of the text (figure 3). It also has some 49 items on the 
side menu bar, 12 on the tool bar menu, seven to 10 
additional information items, 16 links on references, 
three links at the bottom of the page on leaflets, an 
online questionnaire and eight links to the 'small print'. 



Table 4 Estimated probabilities of a naive user randomly 
finding online cognitive behavioural therapy (CBT) from four 
websites where a knowledgeable user could find CCBT in 
three clicks 



Website 



Number of choices 
available at each level 
(click) 

Level 1 Level 2 Level 3 



Probability 
of finding 
online free 
access CBT 



RCPsych 7 20 1 in 140 

http://www.rcpsych.ac.uk/mentalhealthinfoforall/problems/ 
depression/depression. aspx 



NHS Choices 21 



15 



1 in 1890 



http://www.nhs.uk/Conditions/Depression/Pages/ 
Introduction. aspx 

Patient.co.uk 12 36 7 1 in 3024 

http://www.patient.co.uk/health/Depression.htm 
Net Dr 24 8 18 1 in 3456 

http://www.netdoctor.co.uk/depression/index.shtml 

Websites are shown in order of ease of finding online free access 
CBT (website names hyperlinked). 



We assumed that our naive user with depression 
having arrived at this page from Google is able to avoid 
the distractions of the peripheral links and just choose 
one of the seven from the body of the text (figure 4) and 
then find 'Free Online CBT resources' which is one of 20 
choices on the second screen (figure 5). 

Other search terms 

Examination of the first page of 10 results for each of the 
next nine search terms resulted in over 60 different web 
addresses from 37 organisations. (With 2 researchers, 9 
search terms and 10 results each, the total could be 
2X9X10=180.) Five of the websites could lead to online 
CBT, and these were among the 10 most frequently 
presented (table 5). Four of the five were those displayed 
for a search on the single word 'depression' (table 3). 

The Samaritans websites (including main website 
and local variations) were only displayed on entering 
the Samaritans search term. (Samaritans as a search 
term was picked up and displayed for our advert as 
a result of our website having Samaritans as one of the 
destination links.) In summary, table 6 shows the likely 
mean number of websites among the first 10 that 
will link to online CBT in three clicks for a knowl- 
edgeable user. 

Probability model 

What, therefore, is the overall probability of finding an 
online CBT website by searching for 'depression'? How 
does this vary between the optimistic scenario of 
a knowledgeable user and a more pessimistic scenario of 
naive user making random choices? We need to make 
a number of assumptions, namely that: 

1 . We can (for the time being) exclude Google Ads from 
the choices. 

2. Users enter one of 10 search terms, in proportion to 
the number of clicks as shown in table 2. 
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Body of the text 



About this leaflet 

This leaflet is for anyone who is or has been depressed We hope it will also be helpful for friends and relatives 

It describes what depression feels like, some of the help that is available how you can help yourself and how to help 
someone else who is depressed It also mentions some of the things we dont know about depression At the end of the 
leaflet there is a list of other places where you can get further information 

Introduction 

We all feel fed up miserable or sad at times These feelings dont usually last longer than a week or two. and they dont 
interfere too much with our Irves Sometimes there's a reason, sometimes not We usually cope ■ we may talk to a friend 
but dont otherwise need any help 
However in depression 

■ your feelings don't lift after a few days - they carry on for weeks or months 



— PnntaWe version 
* Login 



Additional 
information 

Order our leaflets 
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Figure 3 Screen shot from Royal College of Psychiatrists 'depression' landing page, showing (added labels) side bar menu, tool 
bar menu and additional information. 



3. Users will be offered 10 sites on the first page by 
Google search. 

4. The number of sites from which online CBT can be 
found by a knowledgeable user is as shown in table 5. 

5. Google results will comprise those shown in table 3 
for the term 'depression' and those shown in table 5 
for the next nine search terms. 

6. Knowledgeable users have probability of 1 of finding 
online CBT from the five websites shown in table 5 
and probability of 0 from other sites. 

7. Naive users will have a probability of finding online 
CBT from those sites calculated by the number of 
options at each level (as demonstrated in table 4) and 
probability of 0 for other websites. 

The probabilities of finding online CBT are summar- 
ised in table 7. A knowledgeable user has a one in three 
chance, but a naive user has chance of just 1/150. These 
are probably overestimates as the search results for other 
entered search terms are unlikely to present more 
websites leading to online CBT. 



times the advert is presented at which a user chooses the 
advert. Our click through rate was 1.5%. The number of 
adverts displayed by Google, either on the right side of 
the search panel or as the first few searches, in the 
sponsored links areas is variable. For our users searching 
for depression, there were typically eight adverts. If we 
assume that a typical search and adverts screen therefore 
displays 10 search results and eight adverts, the user has 
an overall choice of 18 items. We might estimate that 
the chance of clicking on any one advert is 1.5% so the 
chance of clicking on an advert is 8Xl.5%=12% and the 
chance of clicking on a Google search result is 88%. 

Box 1 shows the Google Ads that were seen by two or 
more of our users, one of which (The Royal College of 
Psychiatrists — RCPsych) has a one in 140 chance of 
reaching online CBT as shown in table 4. If we assume 
therefore that in addition to the 10 Google searches 
each user is presented with eight Ads, one of which is 
http:/ /www.rcpsych. ac.uk, then the probabilities of 
reaching online CBT, using figures from table 7, become 



Change in probabilities through addition of Google advert 

What happens to these probabilities if we take into 
account Google Ads? We have no data on the relative 
chance of someone clicking on a 'paid for' Google Ad 
rather than a Google search result. However, we can 
estimate from our own Google Ads campaign by using 
the 'click through rate'. This refers to the number of 



Knowledgeable user : (0.88X0.33) + (0.12X1/8X1/140) 
= 0.2904 + 0.00011 = 0.2905 

Naive user: (0.88X0.0069) + (0.12X1/8X1/140) 
= 0.0061 + 0.00011 = 0.0062. 
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best things to do and try them 

■ Keep hopeful 

Remind yourself that: 

■ Many other people have had depression. 

■ It may be hard to believe, but you will eventually come out of it. 

■ Depression can sometimes be helpful - you may come out of it stronger and better able to cope It can help you to see situations and 
relationships more clearly 

■ You may be able to make important decisions and changes in your life, which you have avoided in the past 
What kind of help is available? 

Most people with depression are treated by their GP Depending on your symptoms, the seventy of the depression and the circumstances, your doctor 
may suggest: 

■ self-help 

■ talking treatments 

■ antidepressant tablets 



Guided self-help 
This can include: 



-Some titles did 
not have links 



Some did 

• Self-help leaflets or books, using Cognitive Behavioural Therapy (CBT) principles 

■ Self help computet programmes or the internet 

■ Exercise • 3 sessions per week for 45 minutes to 1 hour, for between 10 and 12 weeks 
Whichever of these is right for you depends on your personality and lifestyle 

Talking treatments 

There are many different sorts of psychotherapy available, some of which are very effective for people with mild to moderate depression They include: 

■ Counselling 

Simply talking about your feelings can be helpful, however depressed you are Sometimes it is hard to express your real feelings even to close friends 
Talking things through with a trained counsellor or therapist can be easier It can be a relief to get things off your chest, and it can help you to be clearer 
about how you feel about your life and other people There may be a counsellor at your GP surgery with whom you can talk, or your GP can refer you to 
a local counselling service - 
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Figure 4 Screen shot from Royal College of Psychiatrists 'depression' landing page, showing link words in the body of the text 
and other headings that were not linked. 



We can now calculate the increase in probability of 
adding an advert that leads directly to online CBT and 
assume that this is one of the eight adverts showing 
alongside that for the RCPsych. 



Figure 5 Section from second screen for http://www. 
RCPsych.ac.uk that has route to CCBT. 



The probability of finding online CBT via an advert 
becomes 

(0.12X1/8X(1/140 + 1)) = 0.0151. 

So probabilities of finding online CBT become 

Knowledgeable user : 0.2904 + 0.0151 = 0.3055 

Naive user : 0.0061 + 0.0151 = 0.0212. 

So adding a Google AdWords advert which leads 
directly to online CBT would improve the chance of 
finding CCBT for knowledgeable users a little (from 
29% to 31%) but would improve the chances of a naive 
user from one in 167 to one in 66. 

DISCUSSION 

Increasingly in UK and elsewhere, people are encour- 
aged to self-refer for support and guidance in using 
online CBT. For example, Increasing Access to Psycho- 
logical Therapies (IAPT) — a major Department of 
Health initiative with delivery across England — is 
promoting self-referral and finding positive benefits. 32 
This is also seen in the national telephone support line 
NHS Living Life. Approaches such as these are likely to 
access a range of people who might not otherwise seek 
help via their GP. 3 Support and guidance are recom- 
mended for all forms of CBT resources — including 



Further reading 

The 'Overcoming 1 series, Constable and Robinson 
A large series of self-help books which use the theori 
overcoming social anxiety and shyness, overcoming i 

Blenkiron P. (2010) Stories and analogies in Cognitiv< 

Free online CBT resources: 

■ MoodGYM: Information, quizzes, games and : 

■ Living Life to the Full: Free online life skills 
and make changes in your thinking, activities 

■ FearFighter: free access can only be prescril 

References 

■ Williams C J & Garland A (2002) A cognitive-b 
Treatment 8 172-179. 

■ Department of Health (2007). Improving access 
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Table 5 The 12 most frequently presented websites (out of 37 in total) from searches by two researchers using nine different 
search terms, showing the number of times presented (out of 180 (=2x9x10) search results) and whether the site can be used 
to find online cognitive behavioural therapy (CBT) in three clicks 



Websites presented by Google in first 10 results 
for two researchers for nine search terms 


Number of times 
presented 


Could online CBT be found 
from this site in three clicks? 


http://www.helpguide.org 




NO 


http://www.samaritans.org 


TO 


NO 


http://www.nhs.uk 


12 


Yes 


http://www.clinical-depression.co.uk 


1 U 


NO 


I I 1 1 [J.// WW W . Uc \J I coo I Ul let I Met I IOC . Ul L) 


1 n 


Ma 
IMU 


http://www.mind.org.uk 


10 


No 


http://www.patient.co.uk 


10 


Yes 


http ://www. rcpsych .ac.uk 


8 


Yes 


http://www.bbc.co.uk 


6 


No 


http://www.get.gg 


6 


Yes 


http://www.netdoctor.co.uk 


6 


Yes 


http://www.overcomedepression.co.uk 


6 


No 


All websites 


180 





We show the main website address rather than the 'start location' within each site. 



online CBT. 34 Although national treatment guidelines 
such as NICE (2009) and SIGN (2010) 19 35 recommend 
the addition of support, many people accessing online 
CBT through free sites will do so without support. 

This study showed that the 3868 people who clicked 
on our advert typically searched with the single word 
depression, although there were a great variety of terms 
entered, including misspellings. However, our study 
demonstrated that even a knowledgeable person may 
have to search quite hard to find online CBT, and it 
would be difficult, or extremely unlikely, for a naive user 
of depression websites (ie, someone not specifically 
searching for online CBT for depression and knowing 
how to find it) to find online CBT websites easily using 
a Google search. Although our estimates use a large 
number of assumptions, we consider that a nearly 
threefold increase in the probability of finding online 
CBT by the addition of a Google Advert is likely to lead 
to more people with depression registering and using 



Table 6 Search terms and the mean number of Google 


search results, out of 10 on the first page, that would offer 


access to online cognitive behavioural therapy (CBT) in 


three clicks 






Google search results 




in first 10 that could 


Search term 


lead to CBT 


depression 


4 


depression self help 


4 


self help for depression 


4 


depression symptoms 


3 


coping with depression 


3 


depression help 


2 


depression test 


2 


help with depression 


2 


dealing with depression 


1 


Samaritans 


0 





online CBT. The real test of this will be if there is an 
increase in uptake demonstrated in our cluster rando- 
mised trial comparing intervention with control areas. 

Based on our 'click through rate' (the number of 
times users clicked on our advert divided by the number 
of times it was presented) of 1.5% and that typically eight 
adverts are displayed on a page of search results, we 
estimated that users may click on an advert in 12% of 
cases when Google displayed search results and adverts. 
Although this is based on our empirical data, online 
adverts may not be viewed positively by internet users 

Table 7 Probabilities of finding online cognitive 
behavioural therapy (CBT) for knowledgeable and naive 
users depending on search term entered 

Probability of 
Probability finding CBT Probability of 
of using this knowledgeable finding CBT 



search term 


user 


naive user 


depression 0.63 


0.4 


0.0083 


depression 0.09 


0.2 


0.0075 


help 






help with 0.05 


0.2 


0.0075 


depression 






depression 0.04 


0.3 


0.0011 


symptoms 






self help for 0.04 


0.4 


0.0083 


depression 






Samaritans 0.04 


0 


0.0000 


depression 0.04 


0.2 


0.0008 


test 






dealing with 0.03 


0.1 


0.0005 


depression 






coping with 0.02 


0.3 


0.0011 


depression 






depression 0.02 


0.4 


0.0083 


self help 






All 1.00 


0.33 


0.0069 
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Box 1 Websites advertised by Google AdWords seen by 
two or more from the panel of 12 participants 



Sites advertised by Google Ads appearing on 
search using keyword 'depression' for two or 
more of the panel 

http://www.fightingdepression.co.uk (11) 
http://www.onlinehelpfordepression.org.uk (6) 
http://www.susanjeffers.com (5) 
http://www.turn2me.org/Depression (4) 
http://www.getconnected.org.uk/Depression (4) 
http://www.cbtwestlondon.co.uk (4) 
http://www.samaritans.org (3) 
http://www.healthyplace.com (2) 
http://www.rcpsych.ac.uk/info (2) 
http://www.aware.ie (2) 
http://www.rethink.org (2) 
http://www.greatvine.com (2) 
http://re-root.com/ (2) 



who are looking for credible information on the web. 
AdWords are sometimes used to promote bogus, 
misleading (eg, 'miracle cures') and malware-ridden 
sites, and many users will avoid them. Our own advert 
(figure 1), mentioning NHS and with a '.org.uk' URL, 
may have had a better click through rate than other 
online adverts, 36 so we may have overestimated the 
impact of online adverts. 

The Department of Health was heavily criticised for 
spending £2.5 million on AdWords between February 
2009 and January 2010. 3 The Department of Health 
declined to give further details of how they had spent 
this money or its cost— effectiveness. By judicious design 
of the website, the NHS Choices and other NHS websites 
should bring high profile search results. In this study, for 
example, the NHS Choices depression section was the 
only website that regularly appeared in the top three 
search results. If people find a website via a normal web 
search, then online advertising such as AdWords may be 
a waste of money. However, this study shows that, if there 
is substantial 'competition' for a user's attention, adver- 
tising may be worthwhile. For example, the Royal 
College of Psychiatrists seems to be spending quite large 
sums on Google AdWords as their advert appears 
frequently alongside their website being found by 
'normal' Google search. For people with depression, 
given that adverts will in any case appear, having an 
advert for RCPsych that gives the user a one in 140 
chance of finding online CBT for depression improves 
their chances very slightly. 

This exploration of how choices may be made given 
the websites and adverts currently being found by 
Google gives some indication that it would be worth- 
while. The number of routes that could be taken, even 
on websites from which it is possible to reach online 
CBT, also shows the bewildering choice facing those with 
depression using the internet. 



Another approach to assessing search experience is 
through observation in a laboratory 3 11 setting or even 
'in the wild', but recruiting, meeting and observing 
people with depression while they search the internet 
presents many ethical and practical difficulties. We were 
able to recruit 12 anonymous users who are likely to have 
searched for depression before, at various locations 
around the UK. They were able to supply us with infor- 
mation of their searches from their natural 'habitat', 
although we restricted their searching to just one search 
term. Ideally, we would ask the panel to search for more 
search terms but we thought this would jeopardise 
getting any response. The results suggested that 
although there was some modification of search results 
by users' previous search history and location, the vari- 
ability was not that great. Although our sample of 12 
users is small, given the lack of variation, it seems 
unlikely that a bigger sample would have identified other 
websites that would have increased the probability of 
finding online CBT. 

Our study is also limited in that we only considered 
Google Search and not, for example, Google Search or 
Google AdWords embedded in partner sites, blogs or 
YouTube videos (the so-called Google Display Network). 
Also, not only do websites change but also Google search 
results change by location, user and over time. Website 
owners are also constantly modifying their websites and 
taking steps to raise their profiles for search engines 
such as Google. Google is also not the only way of 
advertising online and others have used, for example, 
paid location and demographic targeted adverts in 
Facebook on. 38 

The approach we have taken to assess the ease of 
finding CCBT could easily be replicated in future. The 
general idea of assessing the probabilities that expert 
and naive users may have in finding a particular website 
is applicable to anyone considering the addition of 
online adverts to promote online services or resources. 
Online adverts may compete both with other adverts as 
well as with the results of organic search, and health 
service providers considering the use of online adverts 
need a strategy that calculates the added (marginal) 
benefit and cost. 

CONCLUSIONS 

More work is needed into how people make choices 
when confronted by different Google search results and 
adverts. Users of Google AdWords would also be advised 
to consider the context in which they are advertising and 
not just the data reported by Google Analytics. The 
results of a search and the additional impact of adver- 
tising should be explored in more detail before 
committing to a long-term Google AdWords advertising 
budget promoting access to healthcare resources. 
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